Test Automation

Overview

test framework.jpg

Introduction to Test Automation from Pekka Klärck

Benefits
- Very easy and fast to create initially
- No programming skill needed
Problems
- Does not test anything unless checkpoints added
- Very fragile
  1. Often single change in UI can break all tests
- Hard to maintain
  1. Plenty of separate test scripts
  2. No modularity or reuse
- System must be ready before automation can start
  1. Does not support acceptance test driven development(ATDD)
Summary
- Seldom a good approach in general
- Never a good basis for large scale automation

Benefits
- Fast to create initially
- Flexible
- Can use common scripting languages
  1. No license costs
Problems
- Creating tests requires programming skills
- Very fragile
  1. One change in the system may break all scripts
- Hard to maintain
  1. Plenty of test scripts
  2. No modularity or reuse
Summary
- Adequate for simple tasks
- Never a good basis for large scale automation

Benefits
- Reuse of code
  1. Creating new tests gets faster
- Maintainability
  1. Changes requires fixes in smaller areas
- Driver scripts are simple
  1. Even novice programmers can understand and edit
  2. Creating new ones is not hard either
Problems
- Building test library requires initial effort
  1. Takes time
  2. Requires programming skills
- Test data embedded into scripts
  1. Requires some understanding of programming
- New tests require new driver scripts
Summary
- Good for simple tasks
- Works also in large usage if everyone who need to understand tests can program
- Not good for non-programmers

Benefits
- Test libraries provide modularity
  1. Same benefits as with modular scripting
- Creating and editing existing tests is very easy
  1. No programming skills needed
- Maintenance responsibilities can be divided
  1. Testers are responsible for the test data
  2. Programmers are responsible for automation code
Problems
- Test cases are similar
  1. For example '1 + 2 = 3' and '1 * 2 = 2'
- New kinds of tests need new driver script
  1. For example '1 * 2 + 3 = 6'
  2. Creating driver scripts requires programming skills
- Initial effort creating parsers and other reusable components can be big
Summary
- Good solution even for larger scale use
- New kinds of tests requiring programming is a minus
- May be an overkill for simple tests

Benefits
- All smae benefits as with data-driven testing
  1. Non-programming can create and edit tests
  2. Separation of test data and code
- Tests can be constructed freely from keywords
  1. Non-programmers can create also new kinds of tests
  2. With suitable keywords also data-driven tests possible
- All tests can be executed by one framework
  1. No need to create and maintain separate driver scripts
Problems
- Initial effort can be really big
  1. But there are open source solutions available
Summary
- Very good solution for large scale use
- Use exiting solutions if you can
- May be an overkill for simple needs